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MAXIMIZING  MDBIIE  SUBSCKIEER  EQUIEMENT  SYSTEM  EFFECTIVENESS 
THRDUGH  EERSCMJEL  SELBCTKaJ  AND  PERPOEMANCE  ASSESSMENT:  AN  INTEKIM  REPORT 


Summary 

The  Mobile  Subscxiber  Equipient  (MSE)  acxjuisition  is  one  of  the  Anty's 
largest  caarrent  procuretnent  programs  and  requires  the  formation  of  three  new 
MOSs;  MSE  Transmission  Systems  Operator  plD) ,  MSE  Network  Switching  System 
Operator  (31F) ,  and  MSE  Oomraunications  Chief  (31W) .  IXie  to  the  and  hi^ 
technology  equipnent,  the  vastly  enhanced  coaranunications  capabilities,  and  the 
training  demands  of  the  MSE  system,  it  is  critical  that  operator  personnel 
selection  be  efficient  and  that  operator  training  be  effective,  '^s  research 
supports  two  major  goals:  (a)  to  identify  personnel  variables  vMch  are 
predictive  of  MSE  (^jerator  performance  to  support  efficient  personnel 
selection  (31D  and  31F) ,  and  (b)  to  develop  quantitative  operator  performance 
criteria  vhich  can  be  used  as  standards  for  training  outccmes  and  Skills 
Qualification  Tests.  A  signal  battalion  trained  and  fielded  with  MSE 
constituted  the  saitple.  Personnel  data  an^yzed  included  ASVAB  sco^  (GT, 

EL,  SC) ,  educational  level,  oonputer  e>g)erienoe,  age,  gender,  time  in  service, 
handedness,  wearing  of  glasses,  MSE  knowledge  t^t  score,  and  training 
evaluation  rating.  Time  and  error  measures  were  collected  on  3 ID  tasks  of 
initializing  a  Radio  Access  Unit  (RAU) ,  loading  Digital  Secure  Voice  Terminals 
(DSVTS) ,  and  affiliating  Group  logic  Units  (GIUs) ;  31F  tasks  measu^  were 
sending  and  receiving  an  Over-the-Air  Rekey  (OTAR) .  Performance  times  and 
error  rates  are  provided,  a  number  of  potential  predictor  variables  are 
identified,  and  prediction  equations  for  most  tasks  are  offered.  These 
findings  will  be  exercised  and  a  broader  range  of  operator  tasks  sou^t  during 
conduct  of  the  MSE  Follow-on  Operational  Evaliiation  (POE)  by  OTEA. 

Introduction 


Background 

The  U.S.  Army  is  increasingly  integrating  hi^  technology  systems  as  it 
r^jgrades  and  modernizes  equipment.  Owing  to  the  rapid  advancement  of 
technology,  new  systems  vtiidh  replace  older  ones  are  placing  greater  demands 
upon  the  soldier,  particularly  of  a  cognitive  nature,  for  the  operation  and 
maintenance  of  state-of-the-art  equipmmt.  These  demands  are  major  concerns 
within  the  Army  relative  to  the  selection  and  training  of  soldiers  vhio  are 
expected  to  effectively  utilize  increasingly  complex  ^tems.  These  concerns 
are  aggravated  by  a  dwindling  proportion  of  service-age  persons  in  the 
population,  an  increasing  demand  for  persons  with  above  average  mental 
abilities,  and  competition  from  the  private  sector  for  workforce.  The  field 
of  communications  is  a  good  example  within  the  Army  of  personnel  supply  and 
demand  constraints  cotpled  with  an  increasingly  complex  technology.  Prime 
examples  of  the  Army's  new  and  more  demanding  comimunications  systems  are  the 
Single  Channel  Ground  and  Airborne  Radio  System  (SINC3GARS)  and  the  ^k±)ile 
S\±iscriber  Equipment  (MSE) . 

In  response  to  concerns  for  personnel  availability  and  hi^  technology 
equipment  training  requirements,  the  Ckanmanding  General  of  the  Training  and 
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Doctrine  Cjoramand  (TERADOC)  in  Nov  87  requested  Amy  Research  Institute  (ARI) 
assistance  in  personnel  selection  and  training  related  to  the  fielding  and 
integration  of  MSB.  MSB  is  a  nondevelcpmental  acquisition  program,  and  as 
such  has  not  evolved  throu^  the  rigorous  research  and  development  sequence 
typical  of  major  new  Anry  systems.  In  essence,  MSB  must  be  made  cperational 
with  minimal  changes  to  equipment  (engineering)  and  force  structure 
(personnel) .  This  requirement  places  a  significant  demand  on  appropriate 
research  organizations  to  provide  information  and  guidance  to  decision  makers 
on  how  to  integrate  and  operationalize  a  major  system  under  a  scenario  of 
appreciable  personnel  and  training  constraints.  ARI,  among  others,  is 
responding  to  this  challenge. 

System  Description 

MSB  is  a  new  battlefield  coraraunications  system  slated  to  became  the 
backbone  of  Amy  corps  and  division  camraunications.  The  system  is  being 
procured  throu^  GIB  and  has  been  fielded  at  Fort  Hood,  Texas  and  Fort 
Gordon,  Georgia.  It  is  anticipated  that  approximately  18,000  soldiers  will 
be  MSB-trained,  with  fielding  throu^  PYOS  to  involve  over  50  signal 
battalions.  It  is  intended  that  this  system  will  constitute  the  mainstay  of 
Amy  battlefield  communications  into  the  21st  century.  Accordingly,  the 
research,  engineering,  and  training  communities  must  sipport  and  ensure  the 
effective  transition,  deployment,  and  utilization  of  the  MSB  system. 

MSB  integrates  the  functions  of  transmission,  switching,  control, 
communications  security,  and  both  voice  and  data  terminal  equipment  into  one 
system.  As  a  switched  telecaramunications  system,  I^B  is  extended  by  mobile 
radiotelephone  and  wire  access.  Table  1  summarizes  the  physical  cortponents 
of  ICB  and  the  major  opjerational  respxDnsibilities  of  each. 

The  heart  of  the  MSB  system  is  node  center  switches  (NCSs) .  These 
centers  provide  connections  to  large  extension  node  switches  (lENS) ,  small 
extension  node  switches  (SBNS) ,  and  radio  access  units  (RAUs)  and  are  linked 
together  by  line-of-si^t  radio  trunks  (IDSs) .  Bxtension  switches  (LENS  and 
SENS)  allow  wire  line  terminal  subscribers  (telephone,  facsimile,  and  data)  to 
enter  the  system.  Radio  access  units  (RAUs)  provide  mobile  radiotelephone 
lasers  an  inteirface  to  MSB  and  the  ability,  throu^  an  NCS,  to  communicate  with 
other  mobile  and  wire  telephone  lasers.  System  control  centers  (SOCs)  provide 
prxxaessing  capability  for  data  irputs  to  aid  in  network  management.  Figure  1 
illustrates  hew  MSB  components  interrelate. 

MSB  subscriber  service,  equipment  of  vhich  is  user-owned,  is  facilitated 
by  digital  nonsecure  voice  terminals  (EWVTs)  and  mebile  subscriber 
radiotelephone  terminals  (IOd?s) .  MSB  is  intended  to  be  capable  of 
interfacing  with  other  ccmraunications  systems,  to  include  combat  net  radio 
(CNR)  users  (SINOGARS) ,  NATO  and  allied  military  systems,  and  host  nation 
commercial  tel^hone  systems.  The  major  conponents  of  MSB  (NCS,  SCC,  lENS, 
SENS,  RAU,  and  IDS)  are  self-contained  assemblages  configured  in  vheeled  vans 
(HMMWV) .  Major  auxiliary  equipment  for  MSB  includes  generators,  15  and  30 
meter  masts,  and  various  antennas.  Detailed  information  on  MSB  system 
components  is  available  in  FM  11-999E,  "Mobile  Subscriber  Equipment  (MSB) 
Architecture. " 
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Table  1 


Riysical  Corcponents  and  Functions  of  MSB  Assemblages 


Conponent 

Function 

Operator 

MOS 

Radio  Access  Unit  (RAU) 

-  Connects  system  throu^  NCSs  vhen 

relocating 

-  Provides  mobile  subscriber  access 

-  Can  function  as  an  MSRT 

31D 

Line-of-Si^t  (IDS)  Radio 

-  Connects  SENS  and  LENS  to  NCS 

-  Connects  node  center  switches 

-  Allows  remoting  of  SENS,  LENS, 

and  RAU 

-  Can  connect  RAUs  to  NCSs 

-  Provides  UHF  ftmctions  required 

at  SENS,  lENS,  and  NCS 

31D 

Small  Extension  Node 

Switch  (SENS) 

-  Interface  with  ccxnbat  net  radios 

-  Coammercial  interface 

-  Serves  unit  command  posts 

-  Provides  access  to  wire  subscribers 

31F 

Large  Extension  Node 

Switch  (lENS) 

“  Interface  with  combat  net  radios 

-  Commercial  interface 

-  Si^jports  larger  concentrations 

of  users 

31F 

Node  Center  Switch  (NCS) 

-  Maintains  list  of  local  subscribers 

-  Hub  of  a  ccstnmo  node 

-  Directly  services  signal  coramarxi 

and  control  elements 

-  Provides  network  switching 

31F/W 

System  Control  Center  (SCC) 

-  Network  management 

-  Connected  to  NCS 

31W 
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Figure  1.  Diagram  of  functional  relationship  of  MSE 
and  user-unit  equipment.  A  typical  node  consists  of  an 
NCS,  5  SENS,  2  RAUs  and  several  LOS  radios,  Is  operated 
by  a  platoon,  and  Is  capable  of  handling  130  wire  and 
40  mobile  subscribers. 
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Purpose  of  Research 


The  purpose  of  this  research  effort  was  to  cxantribute  towards  Antr/  needs 
for  information  and  guidance  relative  to  MSE  operator  personnel  selection 
strategies,  operator  performance  standards,  and  training  requirements.  It 
deals  specifically  with  July  1989  ART  research  on  MSE  operator  performance  and 
training  decay  utilizing  personnel  from  an  MSE-fielded  signal  battalion  at 
Fort  Gordon,  Georgia.  This  research  expands  on  preliminary  findings  of  the 
MSE  Follow-on  Operational  Test  and  Evaluation  (POTE)  conducted  at  Fort  Hood, 
Texas  during  the  summer  and  fall  of  1988. 

Ejploratory  personnel  and  performance  data  collected  under  the  auspices 
of  the  MSE  POTE  and  contributed  by  the  Army  Operational  Test  and  Evaluation 
Agency  (OTEA)  were  reported  by  Buckalew,  Sroootz,  Glaze,  and  Sanders  (1989) . 
Data  collected  at  Fort  Gordon  and  reported  herein  are  based  on  directions 
suggested  by  the  earlier  POTE  data  analysis.  It  is  cautioned  that  the 
findings  of  this  report  are  interim.  Hcwever,  these  findings  suggest  specific 
data  collection  efforts  intended  for  the  MSE  Follcw-on  Operational  Evciluation 
(POE)  scheduled  for  the  2nd  QTR  90.  Specific  attention  is  directed  at  31D  and 
3 IF  MOSs.  Incorporating  viable  findings  from  the  MSE  POE,  composite  results 
can  be  translated  into  suggested  MSE  personnel  selection  strategies,  to 
include  reslotting  of  current  corarnunications  MOS  soldiers,  suggested 
performance  standards  (end  of  training  and  SQT)  for  MSE  MOS  soldiers,  and 
integration  into  the  Electronic  Proving  Ground's  (EPG)  ccaiputer  simulated 
model  of  ICE  system  performance.  Primary  responsibility  for  personnel 
classification  strategy  iiput  resides  with  the  API  Fort  Gordon  Field  Unit,  and 
responsibility  for  deriving  c^)erator  performance  algorithms  for  integration 
into  the  EPG  MSE  model  resides  with  the  API  Fort  Hood  Field  Unit.  Suggestions 
for  MSE  cperator  performance  standards  are  shared  responsibilities  of  these 
two  AEU!  Field  Units. 


Method 


Operators 

Personnel  and  performance  data  were  provided  throu^  OTEA  for  280 
soldiers  transitioned  into  31D,  31F,  and  31W  MOSs  during  the  conduct  of  the 
earlier  MSE  POTE.  Related  data  were  obtained  from  44  soldiers  transitioned 
into  31D  and  31P  MOSs  at  Fort  Gordon.  The  POTE  groi;^)  had  159  transitioned 
into  31D  and  88  transitioned  into  31F,  86%  of  Whom  were  men.  The  Fort  Gordon 
grov^)  (PG)  involved  22  transitioned  into  the  31D  and  22  transitioned  into  the 
31F  MOSs,  71%  of  vhom  were  men.  Peeder  MOSs  for  the  POTE  group  were;  25B, 
31C,  31G,  31K,  31L,  31M,  31N,  31Q,  31V,  31Y,  31Z,  36C,  36L,  36M,  and  72E.  For 

the  PG  grocp,  feeder  MOSs  included  31C,  31K,  31L,  31M,  31N,  and  36M. 

Table  2  shows  the  distribution  of  feeder  MOSs  for  the  PG  groi:p.  In  both 
POTE  and  PG  grotps,  the  largest  preportion  of  soldiers  (49%  and  60%)  came 

from  the  3IM  feeder  MOS.  For  the  POTE  grotp,  ages  rang^  from  19  to  45  years, 

time  in  service  ranged  from  1  to  20  years,  and  education  ranged  from  9  to  17 
years.  Por  the  PG  groip,  ages  ranged  frm  19  to  37  years,  time  in  service 
ranged  from  1  to  17  years,  and  education  ranged  from  11  to  16  years. 
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Table  2 


Distribution  of  MSE  Feeder  MOSs  within  the  Fort  Gordon  Sample 


Prior  MDS 

31D  (1^22) 

31F  (1^22) 

31C 

9.5% 

9.1% 

3IK 

14.3% 

4.5% 

31L 

9.5% 

13.6% 

31M 

66.7% 

54.5% 

31N 

4.5% 

36M 

13.6% 

ASVAB  standardized  test  scores  ranged  frcan  59  to  155  on  the  GT  scale  and  76  to 
141  on  the  EL  scale  for  FOTE  soldiers  and  90  to  133  on  both  scales  for  the  PG 
groip.  Hie  PG  group  ranged  from  79  to  128  on  the  SC  scale  (data  not  available 
for  FOTE  group) .  For  the  FOTE  group,  90%  and  for  the  PG  groi:p,  86%  were 
ri^t-handed.  In  the  PG  grovp,  28%  wore  glasses  (data  not  available  for  the 
FOTE  grocp) .  Hiese  data  suggest,  allowing  for  a  reduced  PG  sample  size, 
similar  sample  characteristics  in  terms  of  personnel  variables.  Table  3 
summarizes  the  personnel  characteristics  of  the  31D  and  31F  soldiers  vho 
constituted  the  Port  Gordon  sample.  Similar  data  for  the  earlier  POTE  grovp 
is  available  (Buckalew  et  al.,  1989). 


Table  3 

Simnmary  Personnel  Characteristics  of  Fort  Gordon  Sample 


Variable 

31D  MOS  (1^22) 

31F  MOS  (1^22) 

Gender 

59%  men 

82%  men 

Mean  Age 

22.6  (19.9-31.0) 

26.5  (19.7-36.6) 

Time  in  Service 

4.1  (1. 1-8.8) 

6.9  (1.3-16.5) 

Education 

12.1  (12-14) 

12.5  (11-16) 

Handedness 

86%  rii^t 

86%  ri^t 

Glasses 

27%  yes 

29%  yes 

ASVAB  GT  Mean 

108.2  (90-130) 

115.5  (98-133) 

ASVAB  EL  Mean 

106.0  (90-124) 

111.4  (95-133) 

ASVAB  SC  Mean 

105.7  (89-125) 

110.3  (79-128) 

MSE  Test  Mean 

40.9%  (23-59) 

53.4%  (27-73) 

Training  Rating 

2.3  (1.4-3. 3) 

2.5  (1.6-3. 8) 

Computer  Es^ierience 

59%  (little  or  no) 

45%  (little  or  no) 

Feeder  MOSs 

31C,  31K,  31L,  31M 

31C,  31K,  31L,  31M,  31N,  36M 
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Instruments  and  Eaulmient 


A  single  RMJ  and  two  NCSs  were  used  to  si:5^rt  c^)erator  task  performance 
testing.  Ancillary  eqtdpment  included  two  diesel-pcwered  generators  and 
necessary  cabling  between  assenoblages.  An  operator  personnel  data  collection 
form  was  administered  to  soldiers  to  obtain  information  on  individual 
backgrounds  exciting  ASVAB  scores.  These  standard  scores  were  obtained  from 
unit  personnel  records.  The  MSE  Rxjwledge  Test,  an  objective  paper-and-pencil 
test  developed  ty  ART,  was  administered  in  a  monitored  classroom  environment 
and  eiiphasized  the  integration  of  31D  and  31F  collective  knowledge.  A 
marpower,  personnel,  and  training  evaluation  questionnaire  developed  by  AEIE 
involving  17  training  items  using  a  5-point  "adequacy"  rating  scale  (strongly 
agree  =  1  throu^  strongly  disagree  =  5)  was  also  administered  in  a  monitored 
classroom  environment.  Clipboards  were  provided  to  each  soldier.  Stopwatches 
were  used  to  obtain  time  measurements  on  operator  tasks  and  structured  data 
collection  forms  were  used  for  recording  time  ard  error  data.  Operators  were 
allowed  to  consult  MSE  technical  manuals  and  any  available  unit  SOPs. 

Procedure 

The  FOTE  data  analyses  constituted  Ehase  1  (POTE)  of  this  research 
project  and  were  intended  to  provide  suggested  directions  for  further 
research.  Data  collection  at  Fort  Gordon  (Phase  2)  was  intended  to  generate 
hypotheses  for  vhich  data  would  be  collected  later.  Phase  3  (POE)  of  this 
project  will  involve  the  collection  and  analysis  of  cperator  personnel  and 
performance  data  suggested  (Phase  1  and  2)  as  relevant  to  personnel  selection 
and  performance  concerns.  Phase  1  data  suggested  potentially  rewarding  areeis 
of  personnel  and  perfontanoe  data  collection.  Phase  2  data  were  intended  to 
generate  hypotheses  as  to  personnel  and  performance  interrelationships,  and 
Phase  3  data  are  intended  to  enpirically  ej^and  and  confirm  or  disconfirm 
hypotheses  on  personnel  characteristics  and  performance.  Respecting  direc¬ 
tions  suggested  by  Phase  2  data.  Phase  3  (POE)  data  collection  and  analyses 
should  provide  valid  personnel  (31D,  31F)  selection  strategies  and  performance 
algorithms  in  terms  of  task  cotpletion  times  and  errors. 

Personnel  and  performance  data  obtained  from  the  MSE  POTE  were  provided 
by  a  contractor  and  without  controls  for,  in  the  case  of  performance,  the 
number  or  identity  of  soldiers  participating  in  a  given  task.  As  a  result, 
interpretation  of  POTE  operator  performance  data  was  limited  by  collection 
techniques.  Data  collection  at  Fort  Gordon  was  acconpli^ed  by  ART  research 
psychologists  and  was  dedicated  to  specific  concerns  with  designated  MSE  tasks 
and  operator  personnel  variables.  While  the  Fort  Gordon  cperator  and  MSE  task 
sairples  were  definitively  smaller  than  those  of  the  ICE  POTE,  the  Fort  Gordon 
data  are  more  valid. 

MSE  POTE  cperator  performance  data  were,  as  discussed  by  Buckalew  et  al. 
(1989) ,  limited  ty  assignment  of  task  performance  times  collectively  to  all 
soldiers  involved.  This  practice  precluded  meaningful  identification  of 
specific  personnel  variables  associated  with  a  MSE  task  performance.  The  Fort 
Gordon  research  effort  delineated  specific  MSE  task  performances  with  specific 
soldiers.  This  allcwed  for  meaningful  associations  between  soldier  personnel 
characteristics  and  selected  MSE  performances  (31D  and  31F  tasks) .  Soldier 
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performances  on  selected  tasks  were  measured  in  a  motorpool  under  controlled 
conditions  (scenarios) .  Data  were  collected  and  recorded  by  ART  research 
psychologists  aided  by  senior  NCD  Subject  Matter  E>5)e3±s  (SMEs)  vho  provided 
information  on  errors  committed  and  the  adequacy  of  task  performance.  Testing 
followed  MSE  training  and  a  unit  FTX.  Prior  to  performance  testing,  personnel 
data,  training  evaluation  ratings,  and  MSE  Kxwledge  Test  data  were  obtained 
in  a  controlled  environment  (battalion  classrocm) .  Soldiers  were  informed  of 
the  purpose  of  the  research. 

Operator  Tasks.  Because  of  constraints  imposed  by  the  small  number  of  data 
collectors  and  SMEs  and  the  availability  of  trained  operators  and  equipment, 
only  a  narrow  range  of  operator  tasks  could  be  assessed  within  each  MSE  MOS 
(31D  and  31F) .  The  31D  task  measured  for  completion  time  and  error  was  to 
initialize  a  Radio  Access  Unit  (RAU) .  Subtasks  for  vhich  discrete  time  and 
error  measures  were  obtained  were;  load  RT-1539S  (8) ,  load  and  affiliate  the 
Digital  Secure  Voice  Terminal  (DSVT) ,  and  affiliate  the  Group  Logic  Unit  (GID) 
to  include  loading  frequency  plans.  For  the  31F  MOS,  the  tasks  of  sending  an 
Over-the-Air  Rdcey  (OTAR)  and  receiving  an  OTAR  were  measured  for  completion 
time  and  error. 

Scoring.  For  performance  testing,  timing  began  vhen  the  operator 
acdoiowledged  understanding  the  assigned  task  and  ended  vhen  the  task  was: 

(a)  successfully  completed  according  to  an  SME,  (b)  terminated  due  to  an 
uncorrected  error  based  on  SME  feedback,  or  (c)  teiminated  due  to  an 
equipment  problem.  The  "Go/No-Go”  evaluations  of  SMEs  are  not  r^xDrted  in 
this  paper  or  used  in  computing  error  rates,  as  operators  often  had  errors 
corrected  by  other  soldiers  or  an  SME  and  were  allowed  to  proceed  with  a  task 
laltimately  rated  "Go"  by  an  SME.  Some  task  times  were  lengthened  by 
operators  correcting  an  error  and  having  to  redo  one  or  more  subtasks. 

Errors  on  a  task  were  coded  "1"  =  No  and  "2"  =  Yes.  The  ICE  Knowledge  Test 
was  administered  and  scored  by  ARI  (Training  Research  lab)  personnel  and  the 
composite  percent  correct  was  entered  into  the  data  base.  The  Training 
Evalxoation  section  (17  items)  of  a  larger  Marpower,  Personnel,  and  Training 
(MPT)  questionnaire  yielded  coded  responses  frcrni  1  (strongly  agree)  throu^  5 
(strongly  disagree) .  A  composite  mean  rating  was  ccmputed  and  entered  in  the 
data  base.  The  Operator  Personnel  Information  form  completed  by  soldiers 
allowed  most  data  to  be  directly  entered  into  the  data  base.  The  wearing  of 
glasses  was  coded  "1"  (Yes)  or  "2"  (No) ,  gender  was  coded  "1"  (male)  or  "2" 
(female) ,  hand  preference  was  coded  "1"  (left)  or  "2"  (ric^t) ,  eind  computer 
experience  was  coded  from  "1"  (extensive)  throu^  "5"  (none) . 


Results 

The  intent  of  this  exploratory  study,  based  on  guidance  provided  by 
exploratory  FOTE  data  analyses,  was  to  identi]fy  meaningful  relationships 
between  selected  personnel  variables,  as  listed  in  Table  3,  and  a  sample  of 
31D  and  31F  operator  task  performances.  In  addition,  data  were  collected  by 
interview  of  MSE  SMEs  ard  administration  of  an  ARI-designed  MSE  Ihcwledge 
Test  to  allow  modeling  and  evaluation  of  MSE  cperator  skill  decay.  Findings 
of  the  skill  decay  research  effort  are  published  separately  (Sabol,  Qiapell, 
&  Meiers,  in  preparation) .  Personnel  and  performance  relationships 
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identified  are  intended  for  more  detailed  evaliaation  and  validation  during 
the  MSE  POE  planned  for  2QTR  IY90 

Table  4  describes  MSE  operator  (31D  and  31F)  performances  on  measured 
tasks  in  terms  of  ccmpletion  time  and  proportion  of  operators  vho  committed  an 
error.  It  must  be  noted  that,  vhile  the  personnel  data  base  in  this  study  \jas 
n  =  22  for  each  (31D  and  31F)  MOS,  sanple  sizes  for  any  specific  task 
performance  ranged  from  n  =  8  to  n  =  13. 


Table  4 

MSE  Operator  Performances  for  Saicpled  Tasks 


Task 

N 

Conpletion  Time 

Mean  and  (SD) 

Operators 
MaJcing  Error 

Initialize  RAU* 

13 

18.7  min 

(6.8) 

39% 

Load  RT-1539S 

12 

2.0  min 

(1.5) 

25% 

Load  DSVT 

9 

1.3  min 

(  .9) 

10% 

Affiliate  GLU 

11 

2.8  min 

(1.7) 

25% 

Send  OTAR* 

9 

8.0  min 

(3.4) 

56% 

Receive  OTAR* 

8 

7.1  min 

(4.7) 

38% 

*RAU  tasks  performed  MOS  31D;  OTAR  tasks  performed  by  MOS  31F. 

Statistically  significant  (p<.10)  Pearson  prxxiuct-mament  correlations 
between  personnel  variables  and  performances  are  listed  in  Table  5.  Given 
this  level  of  probability,  the  matrix  of  31F  personnel  and  performance 
variables  (12  and  4  respectively)  would  be  expected  to  produce  4.8  significant 
correlations,  and  5  were  realized.  For  the  31D  matrix  of  96  personnel  and 
performance  variable  ccarbinations,  9.6  significant  correlations  were  expected 
and  11  were  realized.  Actual  correlation  significance  levels  ranged  from  .085 
to  .001  in  both  matrices,  and  a  majority  (81%)  of  correlation  coefficients 
were  significant  if  a  .06  level  of  significance  was  applied. 
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Table  5 


Significant  Relationships  (p  <  .  10) :  Personnel  and  Performance  Variables 


Task 

N 

Performance  Variables  (r) 

Initialize  RAU  Time 

13 

Age  ( . 61) ,  Time  in 

Service 

(.70) 

Load  Rr-1539s 

12 

None 

Load  DSVT 

9 

MSE  Kicwledge  Test 

(.78) 

Affiliate  GIIJ 

11 

Age  (.62),  Time  in 

Service 

(.57) 

Error  Initializing  RAU* 

13 

Age  (.50) ,  Time  in 

Service 

(.55) ,  ASVAB  EL 

(-.58) ,  ASVAB  SC 

(-.69), 

Glasses  (-.84) 

Error  loading  RT-1539s* 

12 

ASVAB  SC  (-.64) 

Error  Loading  DSVT* 

10 

None 

Error  Affiliating  GIU* 

11 

None 

Send  OTAR  Time 

9 

None 

Error  Sending  OTAR* 

9 

ASVAB  EL  (.77) 

Receive  OTAR  Time 

8 

Age  (-.69),  Time  in  Service  (-.70),  Hand 

(-.74) 

Error  Receiving  OOCAR* 

8 

Glasses  (-.75) 

*  Errors  were  coded  ''1''  =  No  and  ”2”  =  Yes 


Acknowledging  the  goal  of  this  researxdi  and  that  prediction  is  an  attertpt 
to  accoxmt  for  the  variabililY  in  one  variable  based  on  knowledge  of  another, 
it  was  of  interest  to  determine  viiat  combination  of  predictors  (personnel 
variables)  could  account  for  a  specified  proportion  of  MSE  task  performance 
variability.  A  criterion  of  75%  of  variance  accountability  was  arbitrarily 
assigned  and  a  SAS  regression  model  (R  square)  was  ^plied  to  all  predictors 
(personnel  variables)  for  each  operator  task  performance  (time  and  error) . 
Table  6  shows  the  outcome  of  this  analysis  considering  all  available 
personnel  variables,  and  Table  7  shows  results  of  this  analysis  considering 
only  those  measures  available  prior  to  MDS  slotting  (excludes  f(BE  Knowledge 
Test  score  and  training  rating) ,  i.e. ,  a  personnel  selection  perspective. 

While  the  data  of  Table  5  suggest  a  number  of  discrete  variables,  and  those  of 
Tables  6  and  7  suggest  criterion-referenced  combinations  of  variables,  for 
prediction  of  selected  MSE  operator  performance,  it  was  acknowledged  that 
these  predictor  variables  are  interrelated. 
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Table  6 


Performance  Variance  Accountability  (75%)  Predictor  Variables 


Task  Measure  Predictor  Cowtoination  *Variance 

Accoimted  for 


Initialize  RAU 

Time 

Time  in  Service  +  ASVAB  EL  + 

MSE  Knowledge  Test  +  Education 

76% 

Error 

Glasses  +  ASVAB  SC 

80% 

Load  Rr-1539s 

Time 

Glasses  +  Time  in  Service  +  Age  + 

ASVAB  EL  +  ASVAB  GT 

91% 

Error 

ASVAB  SC  +  Education  +  Time  in  Service 

76% 

Load  DSVT 

Time 

Error 

MSE  Kiowledge  Test  +  Age 

N/A 

89% 

Affiliate  GIU 

Time 

Training  Rating  +  Age  +  ASVAB  GT 

88% 

Error 

ASVAB  EL  +  Hand  +  Time  in  Service  +  Age 

82% 

Send  OTAR 

Time 

MSE  Kiowledge  Test  +  ASVAB  GT  + 

Conpater  Experience 

95% 

Error 

ASVAB  EL  +  ASVAB  GT 

97% 

Receive  OTAR 

Time 

Glasses 

81% 

Error 

Glasses 

100% 

*  The  SAS  multiple  regression  program  treatment  of  missing  values  for  a 
subject  on  any  variable  causes  that  subject's  data  on  all  variables  to  be 
disregarded.  Hence,  Ns  for  Table  6  data  may  be  less  than  Ns  for  Table  5  data. 
Thus,  the  variance  accounted  for  as  listed  for  a  predictor  in  Table  6  or  7 
may  not  be  equal  to  the  square  of  the  correlation  for  that  variable  as  listed 
in  Table  5. 
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Table  7 


Performance  Variance  Accountability  (75%)  Predictor  Variables  for  Selection 


Task 

Measure 

Predictor  Combination 

Variance 
Accounted  for 

Initialize  RACJ 

Time 

None 

N/A 

Brror 

ASVAB  SC  +  Glasses 

79% 

Load  ET-1539S 

Time 

Age  +  Time  in  Service  +  ASVAB  EL 
+  ASVAB  GT  +  Glasses 

87% 

Brror 

Age  +  Education  +  ASVAB  EL  +  ASVAB  SC  86% 

Load  DSVT 

Time 

ASVAB  EL  +  ASVAB  GT 

84% 

Error 

Age  +  Time  in  Service  +  ASVAB  SC 

79% 

Affiliate  GIU 

Time 

Gender  +  Age  +  Time  in  Service 
+  Education  +  ASVAB  EL 

78% 

Brror 

Age  +  Time  in  Service  +  Hand 
+  ASVAB  EL  +  ASVAB  GT 

90% 

Send  OTAR 

Time 

Gender  +  Time  in  Service 
+  ASVAB  GT  +  ASVAB  SC 

92% 

Error 

ASVAB  EL  +  ASVAB  GT 

97% 

Receive  OTAR 

Time 

Glasses 

81% 

Error 

Glasses 

100% 

The  goal  of  this  eJ5)loratory  research,  beyond  identifying  individual  and 
ccambination  MSB  performance  predictor  variables,  was  to  provide  insist  for 
MSB  operator  personnel  selection  strategies  and  performance  algorithms.  The 
desirable  outcome  was  tentative  operator  performance  prediction  equations  for 
those  tasks  selected  for  stun^.  To  satisfy  this  requirement,  stepwise 
multiple  correlation  techniques  were  applied  to  obtain  the  most  effective 
ccmibination  of  personnel  variables  in  predicting  task  time  and  error 
performance  within  this  sample.  The  criteria  that  a  predictor  variable  must 
have  a  significance  level  of  p<.50  for  entry  into  the  model  and  an  independent 
prdDability  level  of  p<.  15  to  be  retained  in  the  model  were  applied  throu^  a 
multiple  regression  program  (SAS  stepwise  regression  procedure) .  Table  8 
summarizes  the  outcome  of  this  statistical  procedure  in  terms  of  prediction 
equations  for  specific  MSB  task  performance  (time  and  error)  using  wei^ted 
predictor  variables.  Table  9  provides  similar  products  thou^  considering 
only  those  measures  available  prior  to  MDS  slotting  to  support  a  personnel 
selection  perspective. 
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Table  8 


Best  Available  Performance  Prediction  Equations* 


Task 

Measure 

Variables  and  Wei^ts 

F 

P 

Initialize  RAU 

Time 

12.26  +  1.93  Time  in  Service 

.46 

8.7 

.015 

Error 

4.39  -  .02  ASVAB  SC  -.66 
Glasses 

.80 

17.9 

<.001 

load  Rr-1539s 

Time 

-2.57  +  .32  Time  in  Service 
+2.13  Glasses 

.38 

2.4 

.153 

Error 

3.81  +  .11  Time  in  Service 
-.45  Education  +  .04  ASVAB 
GT  -.05  ASVAB  SC 

.88 

10.9 

.007 

Load  DSVT 

Time 

-.62  +  .05  Age  +  .03  MSE 
Knowledge  Test  -.29 
Training  Rating 

.96 

32.2 

.003 

Error 

N/A 

— 

— 

— 

Affiliate  GIU 

Time 

-13.42  -  .88  Gender  + 

.31  Age  -  .04  MSE 

Knowledge  Test  +  .17  ASVAB 
GT  -  2.84  Training  Rating 

.97 

31.1 

.003 

Error 

-.33  +  .39  Education 

.26 

2.8 

.133 

Send  OTAR 

Time 

1.38  +  .11  KBE  Knowledge 

Test 

.25 

1.7 

.254 

Error 

-1.46  +  .08  ASVAB  EL  - 

.05  ASVAB  GT  +  .17  Glasses 

.99 

85.3 

.002 

Receive  OTAR 

Time 

24.02  -  .02  Time  in  Service 
-2.57  Education  -.03 
Training  Rating  -  7.89 
Glasses  +  2.47  Corrputer 
Experience 

1.00 

>999 

<.001 

Error 

3.00  -  1.00  Glasses 

1.00 

>999 

<.001 

*  Variables  itiust  meet  .50  significance  level  for  entry  and  .15  significance 
level  for  retention  in  model. 
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Table  9 


Best  Available  Performance-Based  Prediction  Equations  for  Selection* 


Task 

Measure 

Variables  and  Wei^ts 

F 

P 

Initialize  RAQ 

Time 

11.80  +  2.00  Time  in  Service 

.48 

10.3 

.008 

Error 

4.30  -  .02  ASVAB  SC 
-  .70  Glasses 

.79 

19.2 

<.001 

Load  Rr-1539s 

Time 

10.00  -  .63  Age  +  1.12  Time 
in  Service  +  1.53  Glasses 

.59 

3.9 

.056 

Error 

6.12  +  .13  Time  in  Service 

-  .63  Education  +  .02 

ASVAB  GT  -  .04  ASVAB  SC 

-  .18  Oanoputer  Experience 

.91 

11.6 

.005 

load  DSVT 

Time 

4.11  +  .34  Age  -  .61  Time  in  .99 
Service  +  .25  ASVAB  EL 
-  .30  ASVAB  GT  -  1.51  Glasses 

62.3 

.003 

Error 

2.28  -  .01  ASVAB  SC 

.11 

1.0 

.342 

Affiliate  GIU 

Time 

-  3.56  +  .27  Age 

.39 

5.8 

.040 

Error 

-  .40  +  .40  Education 

.27 

3.3 

.104 

Send  OTAR 

Time 

1.67  +  .23  Age 

.17 

1.1 

.351 

Error 

N/A 

— 

— 

— 

Receive  0G2AR 

Time 

22.63  -  .03  Time  in  Service 
-  2.44  Education  -  7.73 
Glasses  +  2.63  Cotrputer 
Experience 

1.00 

>999 

.001 

Error 

3.00  -  1.00  Glasses 

1.00 

>999 

<.001 

*  Variables  must  meet  .50  significance  level  for  entry  and  .15  significance 
level  for  retention  in  model. 


Information  was  available  on  an  additional  potential  predictor  variable 
of  prior  (feeder)  MDS.  As  a  major  concern  in  reslotting  considerations,  prior 
MOS  was  ejqjlored  throu^  regression  analysis  techniques  (SAS  general  linear 
models  procedure)  to  determine  its  iirpact  on  ICE  operator  performances.  Given 
any  significant  (p<.05)  F  value  for  the  effect  of  prior  MDS  on  ICE  performance 
(time  and  error) ,  t  tests  were  applied  to  ccrapare  prior  (feeder)  MDS  grocps  on 
MSE  task  performances  (means) .  For  31F  MOS  tasks  (time  and  error  measures  of 
send  and  receive  (DIARs) ,  there  were  no  differences  among  prior  MDS  groi^js. 

For  31D  tasks  (time  and  error  measures  of  initializing  a  RflU  and  subtasks  of 
loading  Rr-1539s,  affiliating  the  GIU,  and  loading  DSVT) ,  only  loading  RT- 
1539s  ^owed  any  differences  among  feeder  MOSs.  Persons  with  a  31K  prior  MOS 
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were  significantly  (p<.05)  faster  than  31C,  31L,  and  3IM  prior  MDS  grovps, 
thou^  they  made  more  errors  (p<.05)  than  31L  and  31M  MDS  groi:5)s. 

Discussion 

MSE  POTE  data,  as  presented  by  Garretson  (1988) ,  were  e>5)lored  by 
Buckalew  et  al.  (1989)  to  suggest  potentially  profitable  directions  for 
personnel  selection  strategy  research  efforts.  Based  on  guidance  from  these 
sources,  the  present  research  effort  was  conducted  to  provide  hypotheses  for 
personnel  selection,  training  standards,  and  cperator  performance  algorithms 
to  be  more  fully  es^lored  during  the  conduct  of  the  MSE  FOE.  It  is  an¬ 
ticipated  that  data  collection  efforts  associated  with  the  POE  will  ej^lore 
the  personnel  variables  identified  herein  as  "predictors”  and  will  ejqjand  the 
task  performance  data  base  for  31D  and  31F  MOS  soldiers  to  include  additional 
"critical"  tasks  such  as  orienting  antennas,  initializing  the  liOS,  NCS,  LENS, 
and  SENS,  and  performing  selected  operational  subtasks  on  each  assemblage. 

Based  on  reported  findings,  there  appears  to  be  valuable  information 
available  for  consideration  by  decision  makers  involved  in  personnel  selection 
and  setting  training  standards.  Respecting  limitations  of  small  samples  of 
both  31D  and  31F  MOSs  and  a  restricted  range  of  cperator  performance  tasks, 
present  findings  suggest  the  efficacy  of  an  expanded  research  effort  to  be 
accomplished  during  the  MSE  POE  to  broaden  the  operator  task  performance  data 
base.  Within  the  small  data  base  of  this  research  effort,  certain  personnel 
characteristics  were  identified  as  potentially  viable  predictors  of  selected 
MSE  task  performances  and  may  deserve  attention  in  reslotting  personnel  to 
accornroodate  MSE  system  requirements.  Prior  to  consideration  of  present 
specific  findings,  limitations  on  these  findings  must  be  clearly  stated  and  it 
must  be  reiterated  that  this  study  was  conducted  as  an  exploratory  effort  to 
identify  variables  and  relation^ips  of  interest  for  future  (FOE)  and  expanded 
study. 

T.1 -mi  tat  ions 

The  operator  personnel  data  base  for  each  MDS  (31D  and  31F)  contained 
information  on  only  22  soldiers.  No  soldier  was  included  in  any  ta^-specific 
performance  data  base  unless  personnel  data  were  available.  Because  of  time 
restrictions  on  personnel  and  equipnent  availability,  not  all  the  22  soldiers 
in  each  MOS  were  tested  for  task  performance;  sample  sizes  for  each  major  task 
(initialize  RAU,  send  OTAR,  and  receive  OTAR)  ranged  from  8  to  13.  Inter¬ 
pretation  of  some  data  (relationships)  must  carefully  attend  to  how  data  were 
scored  or  coded  for  computer  entry,  as  identified  in  the  Scoring  subsection  of 
the  Method  portion  of  this  report.  Also,  the  influence  of  SME  evaluators'  and 
other  soldiers'  prcmpts  on  tested  cperators'  time  and  error  performances  is 
not  fully  kncwn.  It  must  be  stated  that,  vhile  all  31D  cperators  were  trained 
on  the  RAU,  an  undetermined  number  had  cperated  only  the  IDS  since  training. 
Beyond  consequences  for  measured  performances  on  the  RAD,  this  fact  has 
appreciable  implications  for  unit  sustainment  training  needs  and  operator 
skill  decay. 
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Fjjdims 


The  follcwing  are  offered  as  tentative  findings  and  csonclvisions  based  on 
the  small  saitples  of  operators  and  tasks  assessed.  All  findings  are  subject 
to  verification  in  future  planned  testing  (POE) . 

o  The  5th  and  95th  percentile  tines  for  aocotrplishing  a  send  OTAR  task 
(31F)  are  2.4  minutes  and  13.5  minutes,  respectively; 

o  The  probability  of  error  in  sending  an  OTAR  is  approximately  55%; 

o  The  5th  and  95th  percentile  times  for  aoocarplishing  a  receive  OTAR 
task  (31F)  are  <1  minute  and  14.8  minutes,  respectively; 

o  The  probability  of  error  in  receiving  an  OTAR  is  approximately  38%; 

o  The  5th  and  95th  percentile  times  for  initializing  a  RAU  (3 ID)  are  7.5 
minutes  and  30  minutes,  respectively; 

o  The  probability  of  error  in  initializing  a  RAU  is  approximately  39%; 

o  The  5th  and  95th  percentile  times  for  loading  RT-1539S  (31D)  are 
<1  minute  and  3.7  minutes,  respsectively; 

o  The  probability  of  error  in  loading  RT-1539S  is  approximately  25%; 

o  The  5th  and  95th  pjercentile  times  for  loading  a  DSVT  (31D)  are 
<1  minute  and  2.8  minutes,  respectively; 

o  The  probability  of  error  in  loading  a  DSVT  is  10%; 

o  The  5th  and  95th  percentile  times  for  affiliating  the  GIU  (31D)  are 
<1  minute  and  5.5  minutes,  respectively; 

o  The  prcd^ability  of  error  in  affiliating  a  GLU  is  25%; 

o  Time  in  service,  ASVAB  SC,  wearing  of  glasses,  ASVAB  EL,  education, 
age,  MSE  Knowledge  Test  score,  training  evaluation  rating,  ASVAB  GT,  and 
handedness  appear  as  potentially  viable  predictors  for  sampled  31D  task 
performances; 

o  Wearing  of  glasses,  ASVAB  GT,  MSE  Knowledge  Test  score,  ASVAB  EL,  and 
ccatputer  experience  appear  as  potentially  viable  predictors  for  sampled  3  IF 
tasks; 

o  Significant  (variance  accountability)  prediction  equations  exist  for 
time  or  error  performance  in  initializing  a  RAU  (3  ID)  and  subtasks  of  loading 
RT-1539S,  loading  DSVTs,  and  affiliating  GIUs; 
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o  significant  (variance  accountability)  prediction  equations  exist  for 
time  and  error  performances  in  sending  or  receiving  an  OTAR  (31F) ; 

o  ASVAB  scores,  particularly  EL  and  SC,  ^pear  to  be  more  viable 
predictors  of  performance  error  than  for  performance  time. 


Present  findings  need  to  be  validated  and  sipplemented  by  the  ejpanded 
operator  task  base  (31D  and  31F)  data  collection  opportunilY  available 
throui^  conduct  of  the  MSE  POE.  OTEA,  as  the  testing  agency,  has  expressed 
strong  sipjport  of  such  an  effort  and  is  sipporting  the  collection  of  ap¬ 
propriate  data.  Firicil  MSE  cperator  performance  data  analyses  will  be  provided 
to  OTEA,  EPG  (for  integration  of  performance  algorithms  in  its  MSE  hardware 
model) ,  and  the  Signal  School  and  Center  (for  consideration  in  initial 
operator  training  standards,  SQT  score  requirements,  and  operator  MDS 
selection  strategies) . 
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